1. Background and Relevant Art
Computer systems and related technology affect many aspects of society. Indeed, the computer system's ability to process information has transformed the way we live and work. Computer systems now commonly perform a host of tasks (e.g., word processing, scheduling, accounting, etc.) that prior to the advent of the computer system were performed manually. More recently, computer systems have been coupled to one another and to other electronic devices to form both wired and wireless computer networks over which the computer systems and other electronic devices can transfer electronic data. Accordingly, the performance of many computing tasks are distributed across a number of different computer systems and/or a number of different computing environments.
In some computing environments, databases are used to manage sets of data. The size of many data sets is relatively small. These smaller data sets can be stored using the resources of a single computer system. However, other data sets are larger. To store a larger data set, the data set is partitioned into a plurality of database partitions and different partitions are then stored at different computer systems. For example, social networking sites can have hundreds of millions of users. These social networking sites can store user related data for their users in partitioned databases that are distributed across many computer systems.
In some environments, sharding is used to partition a database. Sharding is a form of horizontal partitioning that splits (partitions) one or more tables by row across multiple databases. Horizontal partitioning reduces index size by using multiple smaller index sizes across a single instance of a schema (or same logical server). Sharding goes further by splitting (partitioning) one or more tables across multiple servers (logical of physical). As such, sharding enables distribution of a larger database over multiple machines, improving performance.
However, sharding is difficult to implement. Sharding typically requires hand-coding by an experienced developer or database administrator. Some mechanisms have been developed to at least in part automate sharding. However, these mechanisms are typically implemented and/or utilized at the application layer (as opposed to the database layer). As such, these mechanisms require complex application coordination to work as intended.
Further, over time, the volume and/or arrangement of data in a database or database partitions can change. As such, a database may need to be re-partitioned to more evenly allocate the use of computer system resources, optimize performance, etc. Re-partitioning can include adding new partitions, removing existing partitions, and transferring data between partitions. Unfortunately, re-partitioning typically requires all affected partitions to be taken offline during the re-partitioning process. As such, during re-partitioning users may be unable to access data for (potentially extended) periods of time.